Quantum Walk and Iterated Quantum Games 
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Iterated bipartite quantum games are implemented in terms of the discrete-time quantum walk 
on the line. Our proposal allows for conditional strategies, as two rational agents make a choice 
from a restricted set of two qubit unitary operations. Several frequently used classical strategies give 
rise to families of corresponding quantum strategies. A quantum version of the Prisoner's Dilemma 
in which both players use mixed strategies is presented as a specific example. Since there are now 
quantum walk physical implementations at a proof-of principle stage, this connection may represent 
a step towards the experimental realization of quantum games. 
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I. INTRODUCTION 



II. PRELIMINARY CONCEPTS 



> 
m 

o 

o 

S3 

^— > 
G 

cd 

CT 1 - 



Quantum Walks (QWs) have captured the attention 
of quantum information theorists mainly because of their 
potential for the development of new quantum algorithms 
|2l> bi lift l5| ■ When two independent quantum walks are 
considered, their joint state may be entangled in several 
ways, and this opens interesting possibilities for quantum 
information processing. Quantum walks have been real- 
ized using technologies ranging from NMR processors @ 
to low-intensitylinear optics experiments [2(. Quantum 
Games (QGs) B ©, Hol . IfO ] constitute another approach 
to quantum information processing where quantum play- 
ers can achieve results not available to classical players 
[UEi. In particular, QGs may provide a new persective 
on the long-standing "public goods" distribution problem 
. [T3|. A recent experiment, involving trained rational 
human subjects, indicates that different cooperation lev- 
els (with respect to the corresponding classical game) are 
to be expected in a quantum version of the Prisioner's 
Dilemma, both in the one-shot case and in an iterated 
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In this work, the QWs and QGs approaches to quan- 
tum information processing are related. This is done by 
using the QW to formulate iterated quantum games in 
which conditional strategies (i.e. strategies that depend 
on the previous state of both players) are naturally imple- 
mented. In section II we briefly introduce the discrete- 
time quantum walk and provide some background mate- 
rial about classical, bi-partite, non-zero sum games. In 
Section III we introduce a simple set of rules which al- 
lows the construction of a quantum game based on two 
quatum walkers. We also discuss the possible strategies 
and present some examples. In Section IV we present our 
conclusions. 



Before establishing a connection between QWs and 
QGs, it is necessary to establish a few definitions and 
provide some reference material on both systems. 



A. Discrete-time quantum walk 



The Hilbert space of a quantum walk on a line is 
composed of two parts, TL = TL X ® TL C . The one-qubit 
"coin" subspace, TL C , is spanned by two orthonormal 
states {|0), |1)}. The spatial subspace, H x , is spanned by 
the orthonormal set of position eigenstates, with 
x E Z labelling discrete sites on a line. We use the symbol 
|| •) to distinguish the position eigenstates from the kets in 
TL C . The evolution is generated by repeated application 
of a composite unitary operator U which implements a 
coin operation, followed by a conditional shift in the po- 
sition of the walker. This shift operation entangles the 
coin and position of the walker [16fl . The quantum walk 
with two walkers A, B [13] takes place in a Hilbert space 
TLab — TLa®TLb, where Ha and TLb are isomorphous to 
TL. After N steps, a pure state characterized by a density 

N 



operator po 
with 



|*(0))(*(0)| evolves to p N = U N p {W) 



U = Q ■ (I O U c ) , 



(1) 



'Permanent address: Institute- de Fi'sica, Universidade Federal do 
Rio de Janeiro, CP. 68528, Rio de Janeiro, 21941-972, Brazil. 



where U c is a unitary operation in Hf 2 , I is the iden- 
tity in Ttf 2 and is a shift operation in Hab- We shall 
be concerned with non-separable coin operations U c , i.e. 
which cannot be written as products of operations local 
to subspaces H.a^'Hb- A general conditional shift oper- 
ation may be expressed in terms of position eigenstates 
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\\x A ,x B ) = \\x A ) ® \\x B ) as 

n = J2 {\\xa + s ( °\x b +s i ° ) )(x A ,x B \\ <B> |00)(00| + 

\\x A + s ( a 1 \xb + s ( b ) )(xa,xb\\ (8 |01> <01 1 + 
+ s^,^ + 4 2) )<^,^b|| ® |10)(10| + 
||au + s^^s +4 3) >(^^b|| ® |11)(H|} • (2) 

The sums are over all discrete sites (xa ,xb) on a plane 

and the integer parameters s AB (i = . . . 3) denote the 
step sizes taken by each walker for each coin state. Note 
that this shift operator is non-separable with respect to 
subspaces H.a and TLb, except in the particular case in 
which the parameters satisfy the relations 

„(0) _ Jl) J2) _ (3) (0) _ (2) (1) _ (3) ( „s 
b A A ' °A A ' b B ~ b B ' b B ~ S B • \°) 

The QW model defined by eqs. Q) and © is our 
starting point for establishing a connection with iterated 
quantum games. Before addressing this issue, we 
summarize some relevant information on non-zero sum 
classical games. 



B. Iterated games and conditional strategies 

In classical, bi-partite, non-zero sum games a gain 
by one agent does not imply a loss by the other. The 
paradigmatic example of this situation is the Prisoner's 
Dilemma (PD) [HI in which each agent is confronted with 
the option to "cooperate" (C) or "defect" (D). Both play- 
ers obtain a "reward" R if they both play C, but if one 
of them deviates and plays D, he obtains the tempta- 
tion T > R. If both players defect, they are penalized 
and lose P, but the worst payoff is obtained by a cheated 
cooperator who gets S. In sum, the PD payoffs obey 

T> R> P> S and 2R > T + S (4) 

so that, on the average, it pays more to defect (T + P > 
R + S) and [D,D] is a Nash equilibrium (NE), i.e. in 
which none of the two players can improve his/her payoff 
through a unilateral change in strategy. On the other 
hand, in the case of [C,C] no player can improve his/her 
payoff without worsening the other player's payoff, so it 
is Pareto optimal (PO). The dilemma consists in the fact 
that the NE is not PO. 

When the PD game is played repeatedly, each player 
has the opportunity to retaliate (reward) the other 
for having played D (C) in the previous encounter. 
Iterated games introduce the possibility of developing 
conditional strategies, and mutual cooperation may 
arise as an equilibrium outcome. In this context, 
a classical strategy is usually characterized by four 
conditional probabilities [pr,Ps,Pt,Pp], where pi is 
the probability of choosing C after having received a 



payoff i — R, S, T or P, respectively. For instance, the 
win-stay, lose-shift or Pavlov strategy [l9j . represented 
by [1,0,0, 1], corresponds to flipping the state when the 
previous payoff is unsatisfactory (P or S) . Similarly, an 
unbiased random strategy is described by [h, k, b] and 
the TFT (tit-for-tat) strategy, in which a player copies 
the previous move of the other player, is represented by 
[1,0,1,0]. 



III. QUANTUM WALK AS A QUANTUM 
GAME 

The essential observation is that the operator f2, de- 
fined in eq. ([2]), connects particular coin states with the 
corresponding payoffs sa,b- When the position variables 
xa,b are associated to the accumulated payoffs, an it- 
erated quantum game may be constructed where each 
player applies a certain strategy acting on the coin sub- 
space, as defined below. The number of games that may 
be defined in this way is huge. For illustrative purposes, 
in what follows we use this prescription to construct a 
quantum version of the iterated PD game. However, it 
is important to emphasize that the connection between 
QWs and QGs is rather general and other classical games 
may be constructed and analyzed along similar lines. 

Consider two agents A (Alice) and B (Bob) which act 
as opponents in an iterated quantum game based on the 
PD. Quantum versions of iterated PD games are obtained 
from the QW by three simple rules: 

i. The coin states of each agent are interpreted as 
|0) — > C (cooperation) and |1) — > D (defection). 

ii. Each agent is allowed to act on his/her own "coin" 
qubit with a unitary operation (a strategy) V a or 
Ub in Ti.® 2 . However, an agent is not allowed to 
alter his/her opponent's state with this operation. 

iii. The position subspace is a quantum register in 
which the accumulated payoff of each agent is 
stored. If Xa is the position operator for Al- 
ice, XaH^a) = her average payoff is 
xa = ti(pX A ), and the same holds for Bob. 

The position registers are updated after each coin oper- 
ation, according to eq. © with the following identifica- 
tions 

„(0) _ (0) _ R (i) _ (2) _ ? 

b A — b B " n - °A — b B — °i 

b T - 8$ = T, = S f = P. (5) 

These relations, according to condition (j4]), imply that 
the shift operator O for the quantum iterated PD game 
is non-separable with respect to subspaces A, B. We as- 
sume that both players start in the position eigenstate, 
\xa,xb) = |0,0) with arbitrary initial coin \cq) € Ttf 2 . 
A pure initial state po = ||0,0)(0,0|| (g> |co)(cq| evolves to 
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Pn after N applications of U, defined in eq. (Q}. Then, 
a measurement of the observables Xa,b determines the 
final payoff for each player. Alternatively, the average 
payoff xa.b — (Xa,b) after N iterations can be used to 
determine who did best. As expected, when partial mea- 
surements of the joint coin state are performed before 
each shift operation, the average payoffs of the corre- 
sponding classical PD game are recovered. At the heart 
of the quantum game is the choice of strategy made by 
each agent, within the restrictions of rule (ii) above. The 
operations Ua or Ub may represent classical-like strate- 
gies, but they may also account for new strategies with 
no classical analog. In this context, it is of interest to 
explore the strategic choices available for both players 
and how do they relate to those available in the classical 
iterated PD game. 



A. Sequential vs. simultaneous games 

Assuming that the first qubit from the left is Alice's 
and the second is Bob's, the possible strategies available 
to Alice are represented by the set of unitary two-qubit 
operations that do not alter the second qubit, 

U A = M00)+ ai |10)](00| + [ a2 |0I)+a 3 |ll)](0I| + 
MOO) + a 5 |10)] (10| + [aa|01) + a 7 \ll)} (llj, (6) 

where the complex coefficients aij satisfy the requirements 
for unitarity of U a- Similarly, the possible strategies 
available to Bob are the set of unitary two-qubit oper- 
ations that do not alter the first qubit 



[60 1 00) 
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6i|01>] <00| 
6 B |11>] <10| 



[62 1 00) 
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h\01)] (01| + 
Mil)] (HI (7) 



with the hi satisfying the requirements for the unitarity 
of Ub- These operations allow for conditional strategies, 
in which a player's action depends on the previous coin 
state of both players. Unconditional strategies result from 
separable coin operations, when Ua and U b both reduce 
to local operations in TL C . 

In the general case, [Ua,Ub] 7^ 0, and the order in 
which the operations are applied makes a difference since 
the coin operation in eq. ([1]) can be constructed either as 
Ub ■ Ua, if Alice moves first, or Ua • Ub otherwise. We 
shall refer to these games, with composite coin opera- 
tions applied in a prescribed order, as sequential games. 
Another alternative is that both agents apply their oper- 
ations simultaneously. This gives rise to a simultaneous 
version of the quantum game in which the coin opera- 
tion U c , a unitary operation in Hf 2 , reflects the strate- 
gic choice of both players. A sequential game with sep- 
arable coin operations is identical with the correspond- 
ing simultaneous game. However, for non-separable coin 
operations, some sequential games cannot be played si- 
multaneously and, conversely, some simultanous games 
cannot be played sequentially. We first discuss in some 
detail the strategies available in sequential games, as they 
are easier to visualize. 
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FIG. f : Circuits representing the coin operation of a Pavlov vs. 
Random quantum game. Left: Alice plays Random and Bob re- 
sponds with Pavlov. Right: Alice plays Pavlov and Bob plays 
Random. These circuits map computational basis states into max- 
imally entangled (Bell) states. 



B. Sequential games 

According to rule (ii) above, the quantum strategies 
available to Alice in a sequential game, are represented 
by unitary operations of the form (|6|) affecting only the 
first qubit. The relation with a classical strategy, defined 
by four real parameters [pr,PSiPt,Pp] with pi £ [0,1], 
is made explicit with the parametrization 



«o 


= e iV « A 


/PR 


a 1 


= e« 


>R 


«2 


= e l *\ 


fPS 


«3 


= e« 


»S 


«4 


= e l * T y 


/Pt 


O5 


= e i( 


) T 


«(i 


= e lVP y 


Jvp 


«7 


= e i( 





Vj - PR 
V 1 - PS 
y i - Pt 
V 1 -VP 



(8) 



where the phases (fi, 9i e [— tv, tt]. A similar parametriza- 
tion holds for the coefficients bi in eq. ([7]). Unitarity 
implies (aside from two conditions relating the phases) 
that 



Pr+Pt = Ps +Pp = 1- 



(9) 



Only classical strategies satisfying eq. ([9]) can be im- 
plemented as unitary operations in sequential quantum 
games. Each of these gives rise to a "family" of related 
quantum strategies. Aside from a global phase, a given 
sequential quantum strategy is determined by up to seven 
real parameters. Families of related strategies require 
fewer parameters however. For instance, quantum ver- 
sions of Pavlov's strategy as played by Alice, [1,0,0, 1], 
are implemented with operators of the form 



U p A 



|00) (00| + e lui 1 11) (01 1 + e"' 2 110) (10| + e W3 101) (11| 



0, U P A 



in terms of three arbitrary phases Vj. If Vj 
reduces to a controlled-NOT (CNOT) operation in which 
Bob's coin is the control qubit [20(. Similarly, Pavlov's 
strategies played by Bob are of the form 



U§ = |00)(00|- 



l |01)(01|- 



|11)(10| 



110)(H| 



and for fij = 0, they reduce to a CNOT operation con- 
trolled by Alice's qubit. These are examples of condi- 
tional strategies, based on non-separable operators. Fur- 
thermore, since [U p , U§] ^ 0, the order makes a differ- 
ence when both players use Pavlov strategies. 

ASan example of a separable strategy, con- 
sider the Hadamard operator H, defined by 
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(|00) +i|01)+i|10) - |ll))/2 (left panel) and the fully 
entangled state ( 1 00) + |ll))/-\/2 (right panel). Our re- 
sults are for a set of unbiased values of the parameters 
which fulfill the PD constrains, eq. ^j, R=—P=l and 
T=—S=2. These results show that the classical situa- 
tion (a tie for unbiased initial conditions) is exceptional 
in the quantum case and in the quantum game different 
initial coin states result in very different outcomes. 

It is illustrative to look at the payoff of one agent vs 
the payoff of the other (xa vs. xb)- In Fig. [3] we show 
the results for two different unbiased initial coin states, 
(| 00) + |11)) /V2 and (|01> + |10» /VI 
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FIG. 2: (color online) Alice's (red) and Bob's (blue) payoffs after 
N = 50 steps as a function of the strategic choice parametrized by 
angles £a,b S [0, tt/4] defined by cos 2 £ = pn for each player. The 
surfaces correspond to two unbiased initial conditions (see text for 
details). 

H\k) = J=(|0) + (-l) fe |l)) for k = 0,1, which has 
been widely used as a coin operation in QW's [l|. In 
the context of QG's, the fact that it generates unbiased 
superpositions of the computational basis states makes 
it useful as a quantum version of the classical Random 
strategy [|>f>|)f]' A sequential quantum game in 
which Alice plays Pavlov and Bob replies with Random, 
is described, for a particular choice of phases, by a coin 
operation U c = (h ® H) ■ , represented by the circuit 
in the right panel of Fig. [TJ 



An example with restricted strategic spaces 

New outcomes are possible when quantum strategies 
are confronted. Consider a restricted strategic space 
in which Pr+Ps = 1j so a players strategy is (aside 
from quantum phases) determined by a single angu- 
lar parameter £ defined by cos 2 £ = pr. For £ = 
(pr = 1) it reduces to Pavlov and for £ = 7r/4, (pr = 0.5) 
to Random. Values of £ e [0,7r/4] result in strate- 
gies which interpolate between Random and Pavlov. If 
the same parametrization is adopted for Bob's strat- 
egy, the resulting two-parameter coin operation (assum- 
ing Alice plays first) is U c = Ub(^b) 1 Ua(^a)- This 
surface, after N = 50 iterations, is shown in Fig. [5] 
for two unbiased initial coin states: the product state 
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FIG. 3: Average cumulated payoffs (xa vs. xb) after 50 itera- 
tions for different initial conditions. In panels (a,b) the initial 
coin is (| 00) + |11)) /V2 and in panels (c,d) (|01) + 1 10>) /V2 
Panels (a, c) show lines of constant £a (red online) and panels 
(b,d) show lines of constant £s (blue online). The extreme 
values of £a and £s are indicated by thick lines. In both cases, 
the other parameter ranges from (a Pavlovian strategy) to 
7r/4 (a random strategy). The inset in panel (b) shows the 
lines for £a = 0,£s = n/20 which intersect at the point P in 
which both agents maximize their average payoffs. 

Note that, independently of Bob's choice of strategy 
Alice must play Pavlov (£a = 0) in order to maximize 
her average return (see panel (a) ) . Bob gets the highest 
average payoff when he adopts an intermediate strategy 
with <£b ~ tt/20, provided Alice plays Pavlov (see panel 
(b) ). In fact, this point (£a,£b) = (0, tt/20) is a Nash 
equilibrium which is also Pareto optimal (point P in the 
inset of Fig. [HJd). As shown in panels (c,d), the same 
Pavlov strategy may result in a maximum or minimum 
payoff for Alice, depending on Bob's choice of strategy. 

As discussed above, not all classical strategies have 
quantum analogs in a sequential iterated quantum game. 
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Clearly, playing always C (or D) is forbidden because it 
leads to non unitary operations. For the same reason, 
the TFT strategy is also forbidden in a sequential quan- 
tum game. However, if the strategic space is be extended 
by considering simultaneous moves of both players, this 
strategy becomes an option. 





Random 


Pavlov 


TFT 


Random 


1, 2 


1,2 


not unitary 


Pavlov 


1,2 


1 


2 


TFT 


not unitary 


2 


2 



TABLE I: Some strategies that can be confronted both in 
sequential (1) and simultaneous (2) quantum games. 



C. Simultaneous strategies 

The case of simultaneous moves is closer to the clas- 
sical situation and allows some new strategies to be im- 
plemented. Let {pft,pg,pT,Pp] define the classical strat- 
egy to be implemented by Alice and \PniPs iPriPp] ^ ne 
one by Bob. A simultaneous quantum game confronting 
quantum versions of two these classical strategies involves 
unitary operations of the form, 

~e lVll \fpJi4 e lvl2 \fp~M ^ 13 \[ph4 ^ Vli \fp^ 

e'^^S e^y/pJpB e^yjf&j. e^y/pfrg 
e lV31 \JptPs eiV32 \JptPs eilf33 \JptPs e ^ 34 tJptPs 

e if»^ p A p B e i V42 ^ p ApB e z V43 ^JpA p B I pApB 

where e tVkl are phase factors and pf' B = 1 — pf ,B . For 
a real U c , e zipkl = ±1, unitarity requires 

(Pi - P A s){P B R ~ P B t) = iP A R ~ PM ~ Pi) = 0, 
{pi - Pi)(PR - Pp) = (pi - Pt)(Ps - Pt) = o, 
(pi-piM-pf) = ip$-pi){p B s -p B P ) = Q. 

In the general case, analogous restrictions involving the 
phases <fki apply- For instance, the game in which Al- 
ice plays Pavlov [1, 0, 0, 1] and Bob simultaneously plays 
TFT [1, 0, 1, 0] is implemented by 

U? T = |00)(00|+e iAl |10)(01|+e lA2 |ll)(10|+e lA3 |01)(ll|, 

in terms of three arbitrary phases A^. In the classical 
version of this game, if both agents start playing C with 
probability 0.5, after N iterations each collects a null av- 
erage payoff N(R + P) = 0. In the quantum game with 
the above mentioned Bell state as initial condition, the 
winning chances are not equal and both players end up 
with positive payoffs. In a similar way, other classical 
strategies may be confronted. The simultaneous scheme 
is closer to classical games but it is limited by condi- 
tions (fT0| . For example, if both players adopt Pavlov, 
they cannot play simultaneously, as some of these con- 
ditions are not satisfied. In Table I, we consider three 
classical strategies and indicate which of them can be 
confronted within the quantum sequential and/or simul- 
taneous schemes. Games confronting TFT vs Random 
may be described by (non-unitary) quantum operations. 
We do not consider these extensions in this work. 



IV. CONCLUDING REMARKS 

We have related general bi-partite iterated quantum 
games to discrete time quantum walks. Several strate- 
gies from classical game theory can be implemented in 
terms of elementary two-qubit quantum gates. Each 
of them gives rise to a family of quantum strategies. 
We give the conditions that must be satisfied so that 
two classical strategies may be confronted, either se- 
quentially or simultaneously, in an interated quantum 
game. Some well-known classical strategies, such as TFT 
can only be implemented in the simultaneous scheme. 
Non-conmuting operations, such as those associated to a 
Pavlov-Pavlov confrontation, can only be realized in the 
sequential scheme. Since the parameter space for these 
quantum strategies is extremely large, instead of a sys- 
tematic exploration, we have shown through selected ex- 
amples that the outcome of a QG may be different from 
that of the classical counterpart. 

In one-shot quantum games, there is a threshold for 
the amount of entanglement in the initial state that al- 
lows quantum features to emerge 0, H3 |. In our pro- 
posal, entanglement is dynamically generated by condi- 
tional operations and the preparation of an initially en- 
tangled state is not required. We have characterized the 
bi-partite entanglement between both agents in a Pavlov- 
Random QG using the von Neumann entropy of the re- 
duced density operator (entropy of entanglement) and 
found that this quantity increases at a logarithmic rate. 
In order to exploit entanglement partial measurements 
may be included as part of the strategic choices. 

The connection between bipartite quantum games and 
discrete-time quantum walks opens the possibility of ex- 
perimentally testing iterated quantum games and strate- 
gies using simple linear optics elements Q. The sensi- 
tivity of these QGs to the choice of the initial state may 
be attenuated in experimental realizations through the 
introduction of decoherence. The impact of a weak cou- 
pling to the environment is a relevant issue in the study 
of quantum games, which deserves further study, as, in 
the classical case, noise-related effects are able to radi- 
cally change the outcome of the different strategies plj ]. 
An initial step in this direction may be considering the 
outcome, for different strategic options, of opposing a 
classical player vs. a quantum player. 

The scheme we have introduced for quantizing the it- 
erated PD game can obviously be applied to 2 x 2 games 
with an arbitrary payoff matrix. There are several pop- 
ular games that seem interesting to analyze within this 
framework. For example the Hawk-Dove (22j , in which 
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the damage from mutual defection in the PD is increased 
so that it finally exceeds the damage suffered by being 
exploited: T > R> S > P. Or the Stag Hunt game 0, 
corresponding to the payoffs rank order R > T > P > S 
i.e. when the reward R for mutual cooperation in the 
PD games surpasses the temptation T to defect. 

Another generalization of this scheme involves multi- 
partite games. The basic evolution, given by eqs. ([T]) 
and ©, may be generalized to accomodate any num- 
ber of quantum walkers. This may be useful for the 
"public goods" problem, since there are indications that 
the quantum version of this multiplayer game may pro- 



vide a more efficient distribution of resources [15| |. How- 
ever, this generalization raises non-trivial issues regard- 
ing the multipartite entanglement which may be dynam- 
ically generated within the game. 
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